Automatic update of connection to a movable object

ABSTRACT

Described herein are systems, methods, storage media, and computer programs that support communication in a movable object environment. In one embodiment, information for a set of communication parameters is provided, a modified value for at least one of the set of communication parameters is obtained from an affiliated device; and a connection established between the movable object and the affiliated device is configured based on the modified value. In another embodiment, a connection between a movable object and an affiliated device is established, wherein the connection uses a set of communication parameters, values of which are set to default values; a modified value for at least one of the set is obtained, based on values for the set of communication parameters provided by the movable object and values for the set of communication parameters provided by the affiliated device, and the connection is configured based on the modified value.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever.

FIELD OF THE INVENTION

The disclosed embodiments relate generally to software applicationdevelopment and more particularly, but not exclusively, to applicationdevelopment for using movable objects.

BACKGROUND

Unmanned aircrafts, as well as other movable objects, such as theunmanned vehicles and robots, have seen rapid adoption in a wide rangeof fields. Movable objects are used in aerial photography/video, parceldelivery, agriculture, rescue mission, and space exploration, and peopleare constantly finding new ways to utilize movable objects as themovable objects often provide gateways to new industries and also offerunique perspective for advancing traditional industries.

Traditionally a vendor of a movable object provides end-to-endsolutions, and the vendor provides both hardware and software in apredetermined field. However, as movable objects move out of the realmlargely occupied by hobbyists and move into mainstream commerce,proprietary solutions are no longer suitable. A closed movable objectenvironment, where the vendor of a movable object offers littleinformation to an independent application developer, hinders theadoption of the movable object in various fields where the vendor hasless insight or less resources than that of the application developer.Yet, the vendor of a movable object should retain a level of control inmanaging the movable object to prevent malicious attack to the movableobject.

It is beneficial to provide enough information to make communicationwith the movable object efficient and allow other developers to developapplications for the movable object, and that is the general area thatembodiments of the invention are intended to address.

SUMMARY

Described herein are systems, methods, storage media, and computerprograms that support application development in a movable objectenvironment, wherein protocol independent movable object application maybe developed. In one embodiment, a request is received in a firstprotocol from an application executing on an affiliated device to accessa movable object; the request in the first protocol is mapped to afunction in an application programming interface (API), where the API isindependent of the first protocol; and the movable object is directed toperform the function, via the API.

Also described herein are systems, methods, storage media, and computerprograms that support data distribution in a movable object environment.In one embodiment, a request for characteristic data of a movable objectis received from a requester of an affiliated device; it is confirmedthat the characteristic data is available on the movable object and thatthe requester has a privilege to access the characteristic data; andthen the characteristic data is provided to the affiliated deviceresponsive to the confirmation. In another embodiment, a request forcharacteristic data of a movable object is transmitted, via acommunication adaptor, from a requester of an affiliated device to themovable object, where the request for characteristic data includesinformation indicative of whether the affiliated device having aprivilege to access the characteristic data; the characteristic datafrom the movable object is received, via the communication adaptor,responsive to the request for the characteristic data; and thecharacteristic data is provided, via the communication adaptor, to therequester of the affiliated device.

Additionally, described herein are systems, methods, storage media, andcomputer programs that support communication in a movable objectenvironment. In one embodiment, information for a set of communicationparameters that are adapted to be used by different affiliated devicesto communicate with a movable object is provided; a modified value forat least one of the set of communication parameters is obtained from anaffiliated device; and a connection established between the movableobject and the affiliated device is configured based on the modifiedvalue for the at least one of the set of communication parameters. Inanother embodiment, a connection between a movable object and anaffiliated device is established, wherein the connection uses a set ofcommunication parameters, values of which are set to default values; amodified value for at least one of the set of communication parametersis obtained, based on values for the set of communication parametersprovided by the movable object and values for the set of communicationparameters provided by the affiliated device, and the connection isconfigured based on the modified value for the at least one of the setof communication parameters.

The embodiments of the present invention provide an open and securemovable object environment, where applications in various protocols fromaffiliated devices may interact with a set of APIs to control a movableobject. Additionally, the movable object may distribute its data to anindependent affiliated device and/or an independent application in anaffiliated device. Furthermore, an affiliated device may negotiate witha movable object to establish a connection with optimized communicationparameters.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an exemplary illustration of an application in a movableobject environment, in accordance with various embodiments of thepresent invention.

FIG. 2 is an exemplary illustration of using an authentication serverfor supporting a security model in a movable object environment, inaccordance with various embodiments of the present invention.

FIG. 3 is an exemplary illustration of using an authentication serverfor supporting multiple applications in a movable object environment, inaccordance with various embodiments of the present invention.

FIG. 4 is an exemplary illustration of supporting a security model in amovable object environment, in accordance with various embodiments ofthe present invention.

FIG. 5 is an exemplary illustration of a process to utilize a protocolindependent API for an application exerting control on a movable object,in accordance with various embodiments of the present invention.

FIG. 6 is an exemplary illustration of a movable object for supportingprotocol independent movable object application development in a movableobject environment, in accordance with various embodiments of thepresent invention.

FIG. 7 is an exemplary illustration of protocol mapping in a movableobject environment, in accordance with various embodiments of thepresent invention.

FIG. 8 is an exemplary illustration of organization of APIs in a movableobject environment, in accordance with various embodiments of thepresent invention.

FIG. 9 is a flow diagram illustrating the operations of a protocolindependent movable object environment, in accordance with variousembodiments of the present invention.

FIG. 10 is a flow diagram illustrating the obtaining of the permissionto perform a function based on the privilege of an application, inaccordance with various embodiments of the present invention.

FIG. 11 is an exemplary illustration of data distribution from a movableobject in a movable object environment, in accordance with variousembodiments of the present invention.

FIG. 12 is an exemplary illustration of details of a movable object andan affiliated device for data distribution from the movable object in amovable object environment, in accordance with various embodiments ofthe present invention.

FIG. 13 is a flow diagram illustrating the confirmation of theavailability of the characteristic data through checking a movableobject database, in accordance with various embodiments of the presentinvention.

FIG. 14 is an exemplary illustration of entries within a movable objectdatabase, in accordance with various embodiments of the presentinvention.

FIG. 15A is an exemplary illustration of data distribution in a movableobject environment according to one embodiment of the invention.

FIG. 15B is another exemplary illustration of data distribution in amovable object environment according to one embodiment of the invention.

FIG. 16 is a flow diagram illustrating operations at a movable objectfor data distribution of the movable object, in accordance with variousembodiments of the present invention.

FIG. 17 is a flow diagram illustrating load balancing during datadistribution of a movable object, in accordance with various embodimentsof the present invention.

FIG. 18 is a flow diagram illustrating operations at an affiliateddevice for data distribution of the movable object, in accordance withvarious embodiments of the present invention.

FIG. 19 is an exemplary illustration of a process to update a connectionbetween a movable object and an affiliated device, in accordance withvarious embodiments of the present invention.

FIG. 20A is an exemplary illustration of communication parameternegotiation between a movable object and an affiliated device andconfigure a connection according to one embodiment of the invention.

FIG. 20B is another exemplary illustration of communication parameternegotiation between a movable object and an affiliated device andconfigure a connection according to one embodiment of the invention.

FIG. 21 is an exemplary illustration of configuring and using aconnection between a movable object and an affiliated device accordingto one embodiment of the invention.

FIG. 22 is an exemplary illustration of a set of communicationparameters, in accordance with various embodiments of the presentinvention.

FIG. 23 is a flow diagram illustrating the operations of a movableobject for automatic update of a connection to the movable object inmovable object environment, in accordance with various embodiments ofthe present invention.

FIG. 24 is a flow diagram illustrating the operations of an affiliateddevice for automatic update of a connection to the movable object inmovable object environment, in accordance with various embodiments ofthe present invention.

FIG. 25 is an exemplary illustration of a movable object, in accordancewith various embodiments of the present invention.

FIG. 26 is an exemplary illustration of an affiliated device, inaccordance with various embodiments of the present invention.

DETAILED DESCRIPTION

The invention is illustrated, by way of example and not by way oflimitation, in the figures of the accompanying drawings in which likereferences indicate similar elements. It should be noted that referencesto “an” or “one” or “some” embodiment(s) in this disclosure are notnecessarily to the same embodiment, and such references mean at leastone.

In figures, Bracketed text and blocks with dashed borders (e.g., largedashes, small dashes, dot-dash, and dots) may be used herein toillustrate optional operations that add additional features toembodiments of the invention. However, such notation should not be takento mean that these are the only options or optional operations, and/orthat blocks with solid borders are not optional in certain embodimentsof the invention. Also in figures, reference numbers are used to referto various element or components, the same references in differentfigures indicate the elements or components having the same or similarfunctionalities.

The description of the invention as following uses unmanned aircraft asexample for movable objects. It will be apparent to those skilled in theart that other types of movable objects can be used without limitation.

In the following description and claims, the terms “coupled” and“connected,” along with their derivatives, may be used. It should beunderstood that these terms are not intended as synonyms for each other.“Coupled” is used to indicate that two or more elements, which may ormay not be in direct physical or electrical contact with each other,co-operate or interact with each other. “Connected” is used to indicatethe establishment of communication between two or more elements that arecoupled with each other. A “set,” as used herein refers to any positivewhole number of items including one item.

An electronic device stores and transmits (internally and/or with otherelectronic devices over a network) code (which is composed of softwareinstructions and which is sometimes referred to as computer program codeor a computer program) and/or data using machine-readable media (alsocalled computer-readable media), such as computer or machine-readablestorage media (e.g., magnetic disks, optical disks, read only memory(ROM), flash memory devices, phase change memory) and computer ormachine-readable transmission media (also called a carrier) (e.g.,electrical, optical, radio, acoustical or other form of propagatedsignals—such as carrier waves, infrared signals). Thus, an electronicdevice (e.g., a computer) includes hardware and software, such as a setof one or more microprocessors coupled to one or more machine-readablestorage media to store code for execution on the set of microprocessorsand/or to store data. For instance, an electronic device may includenon-volatile memory containing the code since the non-volatile memorycan persist code/data even when the electronic device is turned off(when power is removed), and while the electronic device is turned onthat part of the code that is to be executed by the microprocessor(s) ofthat electronic device is typically copied from the slower non-volatilememory into volatile memory (e.g., dynamic random-access memory (DRAM),static random-access memory (SRAM)) of that electronic device. Typicalelectronic devices also include a set or one or more physical networkinterface(s) to establish network connections (to transmit and/orreceive code and/or data using propagating signals) with otherelectronic devices.

A movable object is an electronic device that includes one or morepropulsion units to propel the movement of the movable object. A movableobject can be an unmanned aircraft, an unmanned vehicle, or a robot. Onecommonality of these movable objects is that no humanpilot/driver/operator aboard to control these movable objects. That is,the movement of the movable object, using the one or more propulsionunits, is controlled through a different electronic device. An unmannedaircraft is also referred to as an unmanned aerial vehicle (UAV), adrone, or an unmanned aircraft system (UAS), all of which are usedinterchangeably referring to the unmanned aircraft herein.

An affiliated device is an electronic device that affiliates with amovable object in a movable object environment. An affiliated device maybe a wearable electronic device, a handheld electronic device, anonboard electronic device of the movable object, or another movableobject. When the wearable electronic device, handheld electronic device,or the other movable object communicates with the movable objectwirelessly, they may be referred to as a remote electronic device, incontrast to the onboard electronic device, which communicates with themovable object in a short physical distance (e.g., less than 1 meter).The referred affiliation between the affiliated device and the movableobject is typically communicatively coupling (through a communicationnetwork) or connecting between the affiliated device and the movableobject (through one or more wireline).

FIG. 1 is an exemplary illustration of an application in a movableobject environment, in accordance with various embodiments of thepresent invention. As shown in FIG. 1, an application 112 (note that anapplication is often referred to simply as an app) in a movable objectenvironment 100 can communicate with a movable object 150 via acommunication network 190. The application 112 is deployed in anaffiliated device 102 according to one embodiment of the invention.

The communication network 190 may be a variety of wireline or wirelessnetworks. A wireline network is used primarily for communication betweena movable object and an onboard electronic device on the movable object(the wireline between the movable object and the onboard electronicdevice includes one or more physical communication links such as copperlines and optical fibers), while a wireless network can be used forcommunication between the wearable electronic device, the handheldelectronic device, or onboard electronic device the other movable objectand the movable object (the wireless signal used in the wireless networkis typically radio waves in various frequencies).

The wireline network may deploy technologies such as the universalasynchronous receiver/transmitter (UART) technology, the controller areanetwork (CAN) technology, and the inter-integrated circuit (12C)technology; and the wireless communication network may deploytechnologies such as wireless local area network (WLAN) (e.g., WiFiTM),Bluetooth, cellular the third/fourth/fifth generation (3G/4G/5G). Thecommunication network 190 uses a set of communication links tocommunicate between the movable object 150 and the affiliated device102, and a link from the affiliated device 102 to the movable object 150is typically referred to as an uplink and a link in the oppositedirection is referred to as downlink. The uplink can be used fortransmitting control signals from the affiliated device 102 to themovable object 150, and the down link can be used for transmitting data(e.g., media or video stream) as well as responses to the controlsignals from the movable object 150 to the affiliated device 102.

Traditionally a movable object vendor provides end-to-end solutions. Thevendor provides the application 112 to be installed in the affiliateddevice 102 (e.g., a remote controller provided by the movable objectvendor to control the movable object), and the application is used tocontrol the movable object 150. There are a variety of approaches toensure the robustness of communication between the affiliated device 102and the movable object 150 as often the movable object vendor has alimited control over the communication network 190. With reliablecommunication between the affiliated device 102 and the movable object150, the traditional movable object environment and vendor's end-to-endsolutions can be efficient.

However, the traditional movable object environment is not scalable. Aspeople find more and more usages of movable objects in various fields incommerce, it is desirable to have an open movable object environment,where a developer independent of the movable object 150 and/oraffiliated device 102 develops the application 112 to control themovable object 150. The benefit of an open movable object environmentincludes that the movable object vendor may be free from exploring thevarious fields and develop applications for those fields, and let thedevelopers specialized in the particular fields to develop applicationsparticularly suitable to those fields. The movable object vendors maythen devote their attention to their expertise: making the movableobject more versatile, agile, and economic.

In an open movable object environment, a movable object vendor mayprovide the movable object 150; an affiliated device vendor may providethe affiliated device 102, which may host a variety of applications fromdifferent application vendors for different purposes; and an applicationdeveloper may provide the application 112. The movable object vendor,affiliated device vendor, and application developer may be independentof each other, thus their products (the provided movable object,affiliated device, and application) are independent of each other. Forexample, the affiliated device vendor may provide only one or more of anonboard electronic device (e.g., a camera/camcorder or a computer), ahandheld electronic device (e.g., a remote controller of the movableobject, a smartphone, or a tablet/notebook computer) and/or a wearableelectronic device (e.g., smart watches, smart clothing, Google® Glass,or electronic device otherwise coupled to a human body) withoutproviding the application to control a movable object. The affiliateddevice may also be another movable object provided by a differentmovable object vendor (obviously the affiliated device may be a movableobject provided by the same movable object vendor too). At least two ofthe movable object, affiliated device, and application are independentfrom each other (e.g., not provided by the same vendor)—otherwise whenall of the entities are depend on each other (e.g., all provided by thesame vendor), there will be less need to have the open movable objectenvironment.

In one embodiment, an affiliated device provides a platform forapplication developers to develop their applications, where the platformmay be utilized by applications developed by the application developersto interact with and control a movable object.

In the open movable object environment, two interrelated issues areimportant to address: (1) how to make a movable object environment openso that the movable object may interact efficiently with an independentaffiliated device and/or an independent application in an affiliateddevice and (2) how to make the open movable object environment secure sothat the movable object are protected from erroneous controls and/ormalicious attacks.

Additionally, an open movable object environment should allow a movableobject to distribute its data to an independent affiliated device and/oran independent application in an affiliated device. Also, an openmovable object environment should allow a movable object to establish aconnection with an independent affiliated device with optimizedcommunication parameters.

Embodiments of the present invention provide solutions to address theseissues.

As a movable object has no human pilot/driver/operator aboard, it isparamount that a movable object environment in which the movable objectoperates is secure and shielded from abuses or attacks by an actual orfaked user. FIG. 2 is an exemplary illustration of using anauthentication server for supporting a security model in a movableobject environment, in accordance with various embodiments of thepresent invention. In a movable object environment 200, anauthentication server 201 is used for supporting a security model. Theauthentication server 201 may be a part of a movable object (not shown),and it may also be an entity separated from the movable object, e.g.,within a standalone electronic device, or within an affiliated device.

An application is registered with the authentication server 201. Theregistration involves a developer of the application providing personalinformation in one embodiment. Based on the information provided duringregistration 210, the application is provided with a set of privileges.

An application 204 within an affiliated device 202 will be assigned toan application key (often referred to as an app key) 205 by theauthentication server 201. The application key 205 may be assigned basedon an application identifier (not shown) for the application when it wasunder development. The application identifier can be a package name inthe Android system or a bundle name in the iOS system for theapplication. In one embodiment, the application identifier is submittedto the authentication server 201 (e.g., through a web portal that inturn direct such information to the authentication server), whichassigns the application key 205 to the application 204. An applicationkey binds uniquely with the application identifier, and each applicationkey is associated with a set of privileges. The application key isgenerated through a cryptographic algorithm (thus the application keymay also be referred to as a cryptographic key of the application), andthe cryptographic algorithm may be symmetric or asymmetric, and acryptographic key may be a public key (for encryption or signatureverification) or private key (for decryption and sign). Thecryptographic algorithm and cryptographic key are known in the art, thusnot detailed herein.

The application 204 may initiate an activation request 212 for accessinga movable object (not shown). The activation request 212 may be arequest for a privilege 208 to access the movable object. Theauthentication server 201 may apply one or more policies 206 on theactivation request 212 in order to determine whether the application 204should be granted a privilege 208 for accessing the movable object. Thepolicies 206 may be defined using a variety of criteria such as amaximum installation number, and/or different roles of the application.Additionally, the privilege 208 and policies 206 may also indicate thetypes of movable object data, movable object functional modules, andmovable object interfaces (e.g., application programming interfaces(APIs)) that a particular application may access. In one embodiment, theauthentication server 201 makes the privilege grant decision based onthe application key 205, which is contained in the activation request212. Note that the authentication server is required to be involved incommunication between the application and the movable object only forthe initial activation. Once the application is activated, theapplication then can communicate with the movable object without theauthentication server's further involvement, until/unless the privilegeof the application changes.

Additionally, it is to be noted that the application 204 may provide adifferent privilege to a user of the application than the one it obtainsfrom the authentication server 201. The user may have a lower privilegethan the application 204 and the application developer has the controlof granting the user's privilege within the privilege it is granted fromthe authentication server 201.

Additionally, the affiliated device 202 may also register with theauthentication server 201. The authentication server 201 may determinethat a particular type of affiliated devices has one level of privilege,and all applications resides in that type of affiliated devices can'thave privileges higher than that level. For example, a smartphoneprovided by an independent smartphone vendor being an affiliated devicemay be assigned to a lower level of privilege than a remote controllerprovided by the movable object vendor. Additionally, the privilege of anaffiliated device may also indicate the types of movable object data,movable object functional modules, and movable object interfaces thatthe particular affiliated device may access.

FIG. 3 is an exemplary illustration of using an authentication serverfor supporting multiple applications in a movable object environment, inaccordance with various embodiments of the present invention. As shownin FIG. 3, an authentication server 301 in a movable object environment300 can be used for handling the activation requests from differentapplications A-B 304-305, for accessing and controlling a movable object350, based on one or more policies 306.

For example, the application A 304, which is assigned with anapplication key A 305, may be granted with a set of privileges, e.g.,the privilege A 308. Another application, the application B 314, whichis assigned with an application key B 315, may be granted with adifferent set of privilege, e.g., the privilege B 318.

In accordance with various embodiments of the present invention, themovable object environment 300 is a movable object applicationdevelopment environment that includes a software development kit (SDK),which is used for supporting the development of software application inthe movable object environment. The SDK may include a securitymechanism, which includes a low level functionality, which is availableto applications provided by all registered developers, and a high levelfunctionality, which is available only to application provided by theproved developers. In order to obtain the high level privilege, anapplicant may need to go through an approval process, which involves thereal name registration (e.g., registration 210 in FIG. 2) andintentional usage statement.

FIG. 4 is an exemplary illustration of supporting a security model in amovable object environment, in accordance with various embodiments ofthe present invention. As shown in FIG. 4, an authentication server 401in a movable object environment 400, which is a movable objectapplication development environment that includes a SDK, may receive anactivation request 412 from an application 404 for accessing andcontrolling a movable object 450.

The authentication server maintains a registration database 411. Theregistration database 411 maintains information regarding variousapplications. For example, the registration database 411 may maintain amaximum installation number of the application, where the number may bepreconfigured or configured dynamically. Then, the authentication server401 updates an activated application count once the application 404 isactivated.

In accordance with various embodiments of the present invention, theauthentication server 401 may deny the activation request if theactivated application count is equal to or more than the maximuminstallation number for the application. Furthermore, the authenticationserver 401 may increase the maximum installation number in response to aprivilege upgrade request for the application, or the authenticationserver 401 may set the maximum installation number to zero to stopillegal, inappropriate, malicious, or malfunctioning applications.

As shown in FIG. 4, an authentication unit 423, based on interactionwith the authentication server 401 (e.g., verifying the privilege of theapplication 404), grants the application 404 a privilege to use one ormore application programming interfaces (APIs) 454 for accessing themovable object 450. For example, the movable object 450 may be anunmanned aircraft, the APIs 454 of which includes one or more of aflight controller API, a navigation API, a basic function API, a datadistribution API, a camera API, a battery API, and a gimbal API.Additionally, the APIs 454 may include a ground station API, which isassociated with the flight controller API, where the ground station APIoperates to perform one or more flight control operations, which isconfigured to be only accessible by an application with a particular setof privileges. Additionally, the authentication unit 423, based oninteraction with the authentication server 401, may grant theapplication a privilege to access functional modules of the movableobjects such as a flight control module, a gimbal module, a batterymodule, a camera module, and a communication module, which are discussedin more details in relation to FIG. 8.

At the beginning stage of the application development, the SDK of themovable object environment 400 may only approve a small number ofapplications, in order to safeguard the application development. Afterthe developer finishing the development, the SDK can check therequirement documents, technique documents, installation documents, andrelated source codes in an approval process. After finishing theapproval process, the SDK can increase the maximum installation numberfor the application in order to upgrade the application.

In accordance with various embodiments of the present invention, theactivation process based on the maximum installation number can be usedto prevent the malicious application or inappropriate application. Forexample, the system can set the maximum installation number to zero inorder to prevent the activation of malicious applications. Additionally,the system can deactivate the application, such as preventing themalicious applications from accessing the SDK.

In one embodiment, the registration database 411 maintains informationregarding the affiliated devices. For example, the registration database411 may maintain the privilege level of an affiliated device 402. Theaffiliated device 402 may be associated with a maximum number ofapplications that can be running on the affiliated device.

As discussed herein, through the involvement of the authenticationsever, the embodiments of the present invention may provide security tothe movable object environment to prevent illegal, inappropriate,malicious, or malfunctioning applications to access the movable object,thus provide a secure movable object environment.

A secure movable object environment provides a tool to open up a movableobject environment as authentication allows a movable object to beprotected from erroneous controls and malicious attacks. Yet in an openmovable object environment, the movable object is to accept controlsfrom applications in various protocols, and accommodating variousprotocols is challenging.

For example, for performing data distribution, a set of APIs may bebuilt, one for each of mobile OS (such as Android, iOS, Windows, andSymbian) and wireline communication (such as UART, CAN, and I2C). For adifferent application, another set of APIs may be built. Because thatmovable objects are used in more and more applications in a variety offields, building and maintaining the ever-growing number of APIs isburdensome in the movable object environment. Additionally, when a newapplication is developed, a new set of APIs needs to be added. Also,when a movable object vendor offers a new product to the open movableobject environment, the movable object vendor needs to upgrade allexisting APIs in various applications so that these applications can beused in the new product while maintain compatible with existingproducts. Such approach with protocol specific APIs is not scalable andhinders the growth of the movable object industry.

In accordance with various embodiments of the present invention,protocol independent APIs are proposed, where an API includes a set offunctions so that a movable object is to perform one or more of the setof function as required by an application utilizing the API. The API isindependent of protocols so that a request may come from an applicationin any one of a plurality of protocols, and the API performs therequired function as intended by the application.

FIG. 5 is an exemplary illustration of a process to utilize a protocolindependent API for an application exerting control on a movable object,in accordance with various embodiments of the present invention. Asshown in FIG. 5, a movable object environment 500 includes a set ofaffiliated devices 502-506 referred to collectively as affiliateddevices 508, a communication network 590, and a movable object 550, eachof which is similar to the similarly referred entities in FIGS. 1-4. Asdiscussed in relation to FIG. 1, at least two of the movable object,affiliated device, and application are independent of each other in oneembodiment of the invention.

Each affiliated device in FIG. 5 includes an application in a differentprotocol, ranging from protocols 1 to k. The applications in differentprotocols are to illustrate that the movable object environment 500 isopen and can accommodate applications in various protocols. Also notethat one affiliated device may accommodate a plurality of applicationsthat operate in different protocols, thus the number of applications andthe number of protocol count may not match (e.g., the affiliated device506 contains application n in protocol k, where n can be bigger orsmaller than k).

In FIG. 5, task boxes 1-3 illustrate the order in which operations areperformed according to one embodiment of the invention. At task box 1,an application, for example the application 1 in protocol 1 at reference512, is activated. In one embodiment, the activation is through anauthentication server such as the authentication server 401, wherein theapplication initiates an activation request for accessing the movableobject to the activation server. The activation is discussed in relationto FIGS. 2 and 4 thus not repeated here.

Once the application is activated, at task box 2, the applicationobtains control for one or more functional modules of the movable objectvia one or more APIs, which are independent of the application'sprotocol (e.g., the one or more APIs are independent of protocol 1, inwhich the application 1 operates). A protocol independent API allows anapplication, such as an application provided by a vendor different thanthat of which provides the movable object, to interact with the movableobject. The one or more APIs are, as discussed herein above in relationto FIG. 4, interfaces to access the movable object, and after obtainingcontrol for the one or more APIs, the application may control themovable object.

At task box 3, the movable object 550 performs functions in the one ormore APIs. The performance of the function may generate data (e.g.,retrieved movable object operation data such as motion data) and/orinformation (e.g., a response to a request from the application), andthe generated data or information may return to the application in theprotocol in which the application operates.

Through the process, the protocol dependent application is able to takecontrol of the one or more functional modules, and use the protocolindependent one or more APIs to cause the movable object to performfunctions, and finally the movable object returns the data and/orinformation to the application in the protocol. Thus, the movable objectis insulated from the plurality of protocols as the movable objectperforms only functions in the protocol independent one or more APIs,thus the control of the movable object is not burdened with updating andmaintaining all the protocol specific APIs.

FIG. 6 is an exemplary illustration of a movable object for supportingprotocol independent movable object application development in a movableobject environment, in accordance with various embodiments of thepresent invention. As illustrated, a movable object environment 600 issimilar to the movable object environment 500, and the same referencesindicate elements or components having the same or similarfunctionalities, thus only the additional details in FIG. 6 arediscussed.

As illustrated, FIG. 6 provides more details of the movable object 550according to one embodiment of the invention. The movable object 550contains a set of APIs 654-656, and each is coupled to a mapping module664. The set of APIs 654-656 are coupled to a movable object controller621, which controls the movable object 550. The movable objectcontroller 621 also interacts with an authentication unit 623 and amovable object database 625 in one embodiment of the invention. Notethat when the movable object is an unmanned aircraft, an unmannedvehicle, or a robot, the movable object controller is referred to as anaircraft controller, a vehicle controller, or a robot controllerrespectively, or collectively as simply a controller. It is to be notedthat while a single mapping module 664 is illustrated in FIG. 6, in analternative embodiment, each API may be coupled to its own dedicatedmapping module to map requests from applications in various protocols toa function in the API.

In accordance with various embodiments of the present invention, themovable object controller 621, the authentication unit 623, the movableobject database 625, the APIs 654-656, and the mapping 664-666, may bepart of a software development kit (SDK), which is used for supportingthe development of software applications in the movable objectenvironment 600.

The movable object controller 621 controls various functional modules ofthe movable object 550 in one embodiment. For example, an unmannedaircraft can include a camera module, a battery module, a gimbal module,a communication module, and a flight controller module. The movableobject controller 621 establishes and/or manages the communicationbetween the movable object 550 and applications 512-516 within theaffiliated devices 502-506 in one embodiment of the invention.

The movable object database 625 contains data produced by and for themovable object 550. For example, the movable object database 625 mayinclude motion data of the movable object 550 (such as the movableobject's velocity, position, altitude, and/or period of time in motion)and other movable object's operational status data (such as batteryusage and/or available data storage space of a processor of the movableobject). The movable object database 625 may include data related to itstechnical details (identification of the movable object, performance ofits processors, etc.). The movable object database 625 may also containdata related to applications and affiliated devices. For example, thedata may include identification data of an application so that once anapplication is authorized to obtain certain types of data, theauthorization is recorded in the movable object database 625 and themovable object controller 621 and/or the authentication unit 623 willallow the types of data to be provided to the application withoutobtaining an authorization for the application repeatedly in thesubsequent request from the application. These data of a movable object,including the motion data, other operational status data, movableobject's technical details, and the data related to applications andaffiliated device are collectively referred to as characteristic data ofthe movable object herein. While some data are installed prior to theoperations of the movable object, others are written during theoperations of the movable object. For example, the movable objectcontroller 621 may write the operational status data including themotion data and the data related to applications and affiliated devicesin real-time (instantaneously as the data being generated by the movableobject) or near real-time (within seconds/minutes after the data beinggenerated), in which case the database is referred to as a real-timedatabase.

It is to be noted while the movable object controller 621 is illustratedas within the movable object 550, in some embodiments one or more of themovable object controller 621, the authentication unit 623, the movableobject database 625, the APIs 654-656, and the mapping 664-666 areimplemented outside of but communicatively coupled to (e.g., through thecommunication network 590 or a different communication network) themovable object 550. Additionally, one or more of these modules may beintegrated into a single module. For example, a movable objectcontroller may contain one or more of the APIs 654-656, mapping 664-666,authentication unit 623, and movable object database 625. In oneembodiment, the movable object controller 621, including theauthentication unit 623, is implemented within a remote controller ofthe movable object 550.

The APIs 654-656 are protocol independent and they perform functions asrequested without being aware of how the functions are invoked. Forexample, mapping 664 maps a plurality of requests in various protocolsto the same function of API 654, thus the mapping 664 shields theprotocol details of the various protocol from the API 654 and results inthe protocol independent API.

FIG. 7 is an exemplary illustration of protocol mapping in a movableobject environment, in accordance with various embodiments of thepresent invention. FIG. 7 contains entities with the same references asthose of FIG. 6, and the same references indicate elements or componentshaving the same or similar functionalities. The movable objectenvironment 700 is divided into two planes: (1) the protocol specificplane 774, where the elements or components within are aware ofdifferent protocols and operate accordingly; and (2) the protocolindependent plane 772, where the elements or components within areunaware of the protocols that control the movable object.

With such a division, the elements and components within the protocolindependent plane 772 are insulated from the changes due to changes inprotocols in which the applications operate. In one embodiment, theseelements and components, generally closer to the control of a movableobject, may be implemented exclusively by a vendor of the movableobject, while the elements and components in the protocol specific plane774 are implemented jointly by the vendor of the movable object and thevendors of the affiliated devices and/or applications. Without thedistraction of implementation details of the variety of protocols thatapplications may operate under, the elements and components within theprotocol independent plane 772 may be implemented more efficiently. Whenan application in a new protocol is to control the movable object, onlythe protocol specific plane 774 requires update for the new protocol,e.g., adding another mapping to the existing APIs, and no change isrequired in the protocol independent plane 772. Such division makes themovable object environment scalable, and thus opens the door to use themovable object in more and more diverse fields.

For example, an onboard electronic device typically interacts with themovable object through communication media different from what is usedfor a remote electronic device: an onboard device typically communicateswith the movable object through one or more wireline (physicalcommunication links such as copper lines and optical fibers), while aremote electronic device such as a handheld electronic device or awearable electronic device communicates with the movable objectwirelessly through radio waves in various frequencies. Because of thesedifferences, a movable object environment often provides different SDKsfor applications hosted by onboard device and remote devices: an onboardSDK for the former and a mobile SDK for the latter. Within the two SDKs,different APIs are defined for the same or similar functions indifferent protocols.

By using the protocol independent API, different SDKs and different APIswithin the SDKs may be avoided. For example, an onboard affiliateddevice 702 and a remote affiliated device 704 may utilize the same SDK,and the same API (e.g., API 654) for exerting the same or similarcontrol of the movable object.

In the protocol specific plane 774, each application communicates withthe mapping module 664 through a communication adaptor such ascommunication adaptors 732-732 in one embodiment of the invention. Acommunication adaptor is a part of a SDK of a movable object environmentin one embodiment of the invention. The communication adaptor causesmessages (e.g., in the form of commands) in the applications in variousprotocols to be packaged in a format that is comprehensible in themovable object environment, before the affiliated device transmits themessages toward the movable object. Conversely, the communicationadaptor will convert messages received from a movable object by theaffiliated device for the applications into messages in theapplications. Thus, the communication adaptor converts messages outsideof the SDK into messages in the SDK and does the opposite in the reversedirection in one embodiment. A communication adaptor may be an interfaceof the affiliated device (e.g., an API of the affiliated device) in oneembodiment of the invention. The interface is needed as a movable objectapplication is often independent of the affiliated device that hosts themovable object applications.

The mapping module 664 is to convert the request in different protocolsinto a function of an API. A request to API function mapping table 750illustrates the conversion. An API 1 function 1 at reference 762corresponds to different requests in different protocols, so that arequest 1 in protocol 1 at reference 742 and a request 1 in protocol 2at reference 743 both maps to the API 1 function 1. The API 1 function 1can be a function to create a connection between an application and themovable object. For example, an exemplary function for connecting anapplication to an unmanned aircraft is OnCreate( ) The function may bemapped to requests in two different protocols: the requests 1 inprotocol 1 and protocol 2 at references 742 and 743 respectively.Similarly, an exemplary function for disconnecting the connectionbetween an application and a movable object is OnDestroy( )to, and thatfunction (e.g., API function 2 at reference 764) may be mapped torequests in two different protocols: the requests 2 in protocol 1 andprotocol 2 at references 745 and 746 respectively. When an API functionis performed, the API function is associated with the applicationcausing the performance. For example, the application's application keymay be mapped to the function. Thus the movable object controller knowswhich application is performing which API function.

It is to be noted that a function in an API sometimes requires a set ofinput parameters, which may not be identical to the parameters providedby the request in a protocol, and the request may include more or lessthan the set of input parameters of the API function. In that case, themapping module 664 provides adjustment to convert the parametersprovided by the request in the protocol to the set of input parameters.For example, the request may indicate a data identifier for the datarequested and the API may require an input parameter for a hash value ofthe data identifier. The mapping module 664 provides the conversionbetween the data identifier and the hash value of the data identifier.For another example, the API may require an input parameter that is notincluded in the request in the protocol (e.g., the registrationinformation of the application), the mapping module 664 provides thatinput parameter (e.g., obtaining the registration information from theauthentication unit 623), along with the information included in therequest for the other input parameters to the API function.

Once the function is performed on the movable object, the data and/orinformation generated may need to be returned to the application thathas caused the function to be performed. That is, the data and/orinformation generated may need to be converted back to be in theprotocol of the application. For that, the mapping 664 may utilize afunction output to application data/information mapping table 752. Forexample, the API 1 function 1 output 782 is mapped to data/informationin protocol 1 and data and/or information in protocol 2 at references792 and 793 respectively. Based on which application causes the API 1function to be performed, the output is mapped to the data and/orinformation in the protocol in which the application operates, beingeither protocol 1 or protocol 2 in this example.

It is to be noted that the API function output is not necessarilyresponsive to a request of a particular application, and the movableobject may initiate data and/or information transmission to anapplication too. In other words, the movable object controller 621 mayperform a function in an API and determine that the resulting dataand/or information needs to be transmitted to an application without theapplication sending a request first. For example, the movable objectcontroller 621 may determine that application 1 in protocol 1 atreference 512 is malfunctioning, thus it performs an API function (e.g.,OnDestroy( ) to disconnect the connection between the application 1 andthe movable object, and the output of the API function (e.g., API 1function 2 output 784) is mapped to data and/or information in protocol1 (e.g., data/information in protocol 1 at reference 795).

In the movable object environment 700, each request from an applicationwill be mapped to a function in one API. When one request can't map toan existing request to the API function mapping table 750, a new entryneeds to be added to map the request to an API function. Similarly, whenan application causes an API function to be performed on the movableobject, and the performance generates data and/or information for theapplication, the generated data and/or information needs to be mappedback to data and/or information in the application. When such mappingdoes not exist, the function output to application data/informationmapping table 752 needs to add a new entry for it. Thus, when a newapplication is created, the mapping module 664 needs to be updated, andthe update includes creating new entries or modifying existing entriesin the request to API function mapping table 750 and/or the functionoutput to application data/information mapping table 752 in oneembodiment. The update of the mapping module 664 may be performed by thevendor of the movable object in cooperation with a vendor of anapplication and/or a vendor of an affiliated device.

As illustrated in FIG. 7, requests for the same or similar performanceare mapping to the same function in the request to API function mappingtable 750. Thus API functions are categories based on their purposes.The API function categorization makes protocol mapping more efficient.However, a movable object environment generally has a plurality of APIs,and it is desirable to organize APIs too. FIG. 8 is an exemplaryillustration of organization of APIs in a movable object environment, inaccordance with various embodiments of the present invention.

As illustrated in FIG. 8, a movable object maintains a set of protocolindependent APIs 812. The protocol independent APIs 812 include APIs fordifferent purposes, and each API is given a unique API identifier (ID)in one embodiment. For example, a control API 802, which is for motioncontrol of the movable object, is set to be API 1; an intelligentnavigation API 804, which is for the movable object navigating itssurrounding, is set to be API 2; a basic function API 806, which is forbasic operations of the movable object (e.g., checking a battery levelbeing sufficient enough to sustain the movement of the movable object);a data distribution API 808, which is for the movable object todistribute data, is set to be API 4; and a camera API 810, which is forthe still image capture or video recording of a camera of the movableobject, is set to be API 5. Thus, with each API identified by a uniqueAPI ID, the mapping module 862 may identify the APIs required for agiven request in a protocol. The APIs may also include a cloud platformAPI for supporting cloud services for the movable object and acommunication API for the movable object to communicate with affiliateddevices and communicating internally within the movable object, theseAPIs also have unique API IDs but they are not illustrated in FIG. 8 dueto figure space constraint.

In operation, an application in a protocol may initiate a request. Therequest may come from an application coupled to one of the communicationadaptors 832-836. The request is sent from an affiliated device throughthe one of the communication adaptors 832-836, and to the mapping module862.

At the mapping module 862, the request is mapped to an API of theprotocol independent APIs 812. Each API in the protocol independent APIs812 is identified with a unique API ID in one embodiment, thus through amapping table such as the request to API function mapping table 750 therequest is mapped to a particular API. Since the mapping module 862provides mapping (1) between requests in various protocols and APIfunctions and (2) between the API function outputs and data/informationin various protocols, the mapping module 862 is also referred to be arouting module that routes messages (including requests,data/information) between applications and APIs.

The application may initiate a plurality of requests for control of afunctional module of the movable object. The plurality of requests maybe mapped to different APIs. As illustrated in FIG. 8, when the movableobject is an unmanned aircraft, the functional modules include a flightcontroller module 842 to manage the flight of the unmanned aircraft, agimbal module 844 to manage support of an onboard electronic device, abattery module 846 to manage battery power consumption, a camera module848 to manage still image capture or video recording, and acommunication module 849 to manage communication between the unmannedaircraft and an affiliated device. While in one embodiment, onefunctional module is accessible through a single API (for example,camera API 810 is dedicated to camera module 848), in an alternativeembodiment, a single functional module is accessible through a pluralityof APIs.

For the control of a functional module, a plurality of functions withina plurality of APIs may be invoked for the application, and the movableobject controller 821 coordinates the performance of the plurality offunctions in one embodiment. Prior to the application taking control ofa functional module, the movable object controller 821 may determinewhether the application has the privilege to access the functionalmodule (e.g., through checking an authentication server), and whetheranother application is in control of the functional module. If theapplication has the privilege to access and there is no otherapplication in control of the functional module, the application isgiven the control of the functional module and perform one or morefunctions in one or more APIs to direct the movable object. If theapplication has the privilege to access and there is another applicationin control of the functional module, the movable object controller 821determines if the application has a higher privilege than the otherapplication. When the application has a higher privilege, the otherapplication is preempted and the corresponding functions in the APIs areno longer performed, and the application takes over the control.Otherwise the application is denied the control of the functionalmodule.

FIG. 9 is a flow diagram illustrating the operations of a protocolindependent movable object environment, in accordance with variousembodiments of the present invention. Method 900 may be performed by amovable object controller (e.g., the movable object controller 621 or821 in FIGS. 6-8) of a movable object.

At reference 902, an application in a first protocol is activated. Theactivation is through an activation request as discussed herein inrelation to FIGS. 2, 4 and 5 thus not repeated here. Once theapplication is activated, a request in a first protocol is received fromthe application executing on an affiliated device to access a movableobject at reference 904.

At reference 906, the request is mapped to a function in an API, wherethe API is independent of the first protocol. Optionally, a permissionis obtained to perform the function based on a privilege of theapplication at reference 908. Then the movable object is directed toperform the function, via the API at reference 910.

The API allows the application provided by a vendor different than thatof which provides the movable object to interact with the movable objectin one embodiment. The API is adapted to accept one or more requestsfrom a second protocol, different form the first protocol, for directingthe movable object to perform the function in one embodiment. Thus, theAPI may accept requests from different protocols to direct the movableobject to perform the same function. The API is one of a plurality ofAPIs for the movable object and is responsible for access of afunctional module of the movable object in one embodiment. Additionally,in one embodiment, each API of the plurality of APIs is identified witha unique identifier.

In one embodiment, the method 900 continues after operations inreference 910, and information or data from the movable object isobtained for the application at reference 912. The information or datamay be responsive to the movable object performing the functiontriggered by the request in the first protocol in one embodiment.

At reference 914, the information or data is mapped to a message in thefirst protocol. In one embodiment, the message is a response to therequest in the first protocol. At reference 916, the message is providedto the application in the first protocol. The application receives themessage through a communication adaptor in one embodiment of theinvention.

FIG. 10 is a flow diagram illustrating the obtaining of the permissionto perform a function based on the privilege of an application, inaccordance with various embodiments of the present invention. Method1000 includes details of the operations in reference 908 of FIG. 9 inone embodiment. Method 1000 may be performed by a movable objectcontroller.

A request in a protocol is received from an application executing on anaffiliated device to access a movable object. It is determined that therequest maps to a function in an API. At reference 1002, it isdetermined whether an application has a privilege to perform thefunction. In one embodiment, the determination is through checking anauthentication server, e.g., the registration of the application in theauthentication server. If the application does not have the permissionto perform the function, the request from the application is rejected atreference 1016.

If the application has the privilege to perform the function in the API,the flow goes to reference 1004, and it is determined whether any otherapplication is performing the function. If there is another applicationperforming the function, the flow goes to reference 1006, otherwise theflow goes to reference 1008.

At reference 1006, it is determined If the application has a higherprivilege than the other application. If the application does, the flowgoes to reference 1008, otherwise the flow goes to reference 1016 andthe request from the application is rejected.

At reference 1008, the movable object is directed to perform thefunction. Afterward, the flow may go to reference 1018, where operationsin references 912-916 are performed.

While the movable object performs the function, the movable objectcontroller may determine whether any other application with a higherprivilege has a new request to perform the same function at reference1010. If not, the movable object continues perform the functionresponsive to the application at reference 1008. Otherwise, theperformance of the function for the application is halted at reference1012. Then at reference 1014, a response to the request is provided tothe application. The response may include data and/or informationgenerated by the movable object while executing the function for theapplication. The response may also indicate the reason of theperformance of the function being halted. The response will be receivedby the application through a communication adaptor in one embodiment ofthe embodiment.

As discussed, in order to provide an open movable object environment,protocol independent interfaces are provided to applications in variousprotocols, where the applications may initiate requests to control amovable object. Through a mapping module, where the differences betweenthe various protocols are abstracted away, the protocol independentinterfaces do not need to understand the differences between the variousprotocols, and just perform the functions in the interfaces and directthe movable object perform the functions. Additionally, the mappingmodule will map the output of the movable object (e.g., output of theAPI functions), which is protocol independent, into a protocol specificresponse or message in the protocol in which the initiating applicationoperates.

Such an open movable object environment simplifies the design of theinterfaces that control the movable object and the movable objectenvironment becomes more scalable. It is to be noted while theapplication programming interface (API) is used as the exemplaryinterface for a movable object, the embodiments of the invention are notlimited to the API, and the embodiments of the invention may apply toother software interfaces through which an application may exert controlover the movable object.

Through the approaches discussed herein above, a movable objectenvironment supports protocol independent movable object applicationdevelopment, and an application from an affiliated device may direct amovable object to perform operations. In a movable object environment,another important aspect is data distribution from the movable object.Data generated in the movable object should be distributed to anaffiliated device if the affiliated device has the privilege to accessthe data, even if the affiliated device is independent of the movableobject.

Through opening up the data distribution to affiliated devices(independent from the movable object or otherwise), a movable objectenvironment enables the affiliated devices (provided by the movableobject vendor or a vendor independent of the movable object vendor) tomonitor the movable object's status and respond accordingly. Thus anaffiliated device, even if it is independent from a movable object, maycontrol the movable object similarly as a controller provided by thevendor of the movable object (e.g. a remote controller dedicated to themovable object and provided by the vendor of the movable object for thepurpose of controlling the movable object). The data distribution thusallows an application to be developed for a field that the movableobject vendor has less expertise than a developer of the application,where the application may utilize the distributed data for analysis orcontrol of the movable object, thus the movable object may be usedefficiently in the field and broaden the usage of the movable object.

FIG. 11 is an exemplary illustration of data distribution from a movableobject in a movable object environment, in accordance with variousembodiments of the present invention. The movable object environment1100 is similar to the movable object environment of FIG. 5, and thesame references indicate elements or components having the same orsimilar functionalities. In FIG. 11, task boxes 1-3 illustrate the orderin which operations are performed according to one embodiment of theinvention.

At task box 1, the movable object 550 receives a request forcharacteristic data of the movable object 550 from a requester of anaffiliated device 502. The requester is an application executing on theaffiliated device, wherein the application initiates the request forcharacteristic data through an application programming interface (API)that allows the application provided by a vendor different than that ofwhich provides the movable object to interact with the movable object inone embodiment. The API is a protocol independent API discussed hereinin one embodiment.

At task box 2A, the movable object 550 confirms that the requester has aprivilege to access the characteristic data of the movable object 550.At task box 2B, the movable object 550 confirms that the characteristicdata is available. The two confirmations may occur at the same time orsequentially (where either confirmation being done first is acceptable).The confirmation of the requester having the privilege to access thecharacteristic data is performed through checking with an authenticationserver in one embodiment. For example, when the requester is anapplication, the confirmation comprises checking whether theregistration of the application in a registration database within theauthentication server (e.g., the registration database 411 of theauthentication server 401) indicates that the application has thenecessary privilege to access the characteristic data.

Then at task box 3, the movable object 550 provides the characteristicdata to the affiliated device responsive to the confirmations.

FIG. 12 is an exemplary illustration of details of a movable object andan affiliated device for data distribution from the movable object in amovable object environment, in accordance with various embodiments ofthe present invention. The functional units illustrated in the movableobject 550 in FIG. 12 are similar to the ones in FIG. 6, and the samereferences in FIG. 12 indicate the functional units having the same orsimilar functionalities as in FIG. 6.

At the movable object 550 side, the movable object controller 621coordinates data distribution from the movable object, and it operatesto confirm the availability of the characteristic data and therequester's privilege to access the characteristic data, and provide thecharacteristic data. In one embodiment, the confirmation of therequester's privilege is performed through the authentication server 401and the registration database 411 of the authentication server 401,operations of which are discussed in relation to FIG. 4 herein above.

The characteristic data is stored in the movable object database 625.The movable object database 625 includes characteristics data such asthe motion data and data related to applications and affiliated devicesof the movable object 550 as discussed herein above in relation to FIG.6.

At the affiliated device 502 side, the requester is the application 512in an embodiment, and the interactions between the application 512 andthe movable object 550 go through a communication adaptor 532.

In one embodiment, the affiliated device 502 transmits the request forcharacteristic data of the movable object from the application 512, viathe communication adaptor 532. The request for characteristic dataincludes information (e.g., an application key) indicative of whetherthe affiliated device having a privilege to access the characteristicdata. Then the affiliated device 502 receives, via the communicationadaptor 532, the characteristic data from the movable object responsiveto the request for the characteristic data. The affiliated device 502then provides, via the communication adaptor 532, the characteristicdata to the application 512.

The confirmation of the requester having the privilege to access thecharacteristic data is performed through checking with an authenticationserver in one embodiment. The confirmation of the availability of thecharacteristic data is through checking a movable object database 625 inone embodiment.

FIG. 13 is a flow diagram illustrating the confirmation of theavailability of the characteristic data through checking a movableobject database, in accordance with various embodiments of the presentinvention. Method 1300 is performed in a movable object environmentaccording to one embodiment of the invention. In one embodiment, method1300 is responsive to a movable object receiving a request forcharacteristic data of the movable object from a requester of anaffiliated device. The movable object database is within a movableobject in one embodiment, and outside of the movable object in analternative embodiment.

Optionally at reference 1302, it is determined whether a version of thedatabase matches a version for the characteristic data requested. Therequest contains information indicative of the version of thecharacteristic data requested. For example, the request may indicate theversion of the characteristic data, or a cryptographic key generatedbased on the version of the characteristic data. The version of thecharacteristic data is then compared to the version of the database, orthe cryptographic keys of the requested data and the database arecompared, and if a match is identified, method 1300 continues toreference 1304, otherwise, an error will be indicated. Upon getting theerror, the movable object will return an error indication to therequester of the affiliated device.

At reference 1304, a data identifier of the characteristic data from therequest for the characteristic data is determined. The data identifiermay be one or more of a field name of the characteristic data, a hash ofthe field name, and a sequential order of a field for the characteristicdata. The data identifier may be in plain text or it may be encrypted.

At reference 1306, the data identifier is used to identify thecharacteristic data in the database of the movable object. If no matchis found, an error will be indicated. Upon getting the error, themovable object will return an error indication to the requester of theaffiliated device.

In one embodiment, the data in the database has a version, and a matchfor the characteristic data requires a version match too. Thusoptionally at reference 1308, a version of the characteristic data inthe database is determined to match a version for the characteristicdata requested, then it is determined that the matching characteristicdata is available in the database, and the identified characteristicdata is the data for the request. Again, if no match is found, an errorwill be indicated. Upon getting the error, the movable object willreturn an error indication to the requester of the affiliated device.

FIG. 14 is an exemplary illustration of entries within a movable objectdatabase, in accordance with various embodiments of the presentinvention. A movable object database 1400 indicates a version of thedatabase, which is 12.1.00. The database version and/or thecryptographic key generated based on the version is matched against theversion and/or the cryptographic key generated based on the version ofthe requested characteristic data in one embodiment of the invention asdiscussed herein above.

Data in the movable object database 1400 is indexed by data identifiers1402, and data has a version numbers 1404 and the values 1406. The dataidentifiers 1402 are the field names of the characteristic data. Inalternative embodiments, the data identifiers may be hashes of the fieldnames (through one or more hash functions applying to the field names)or a sequential orders of fields (e.g., field 1 corresponds to velocity,field 2 corresponds to position, etc.). In one embodiment, there is noseparate data version information such as the data version number 1404,rather, the data version information is embedded within the dataidentifiers 1402. For example, data identification of data and its dataversion together form a data identifier, thus for the same data type(e.g., velocity), the data identifiers are different when the dataversions are different.

In this example, the data includes the movable object's velocity inmeter per second 1410, position in GPS coordinate 1412, altitude inmeters 1414, and period in motion in minutes 1416. The data versions ofthe data are different for the illustrated data items. In oneembodiment, the versions between the requested data and thecharacteristic data in the database needs to match exactly. For example,if the requested data of a request is for version 12.1.00, the movableobject velocity, which is in version 11.4.00, is not a match, thus eventhough the movable object velocity data is in the movable objectdatabase 1400, the data is determined to be unavailable for the request.In an alternative embodiment, the earlier version of data in thedatabase is considered compatible to the requested later version data,thus the velocity data at version 11.4.00 is compatible to the requestedversion 12.1.00 (often referred to as backward compatible) and the value2 meter/seconds are provided to the requester of the data. Typically, alater version of data in the database is not considered compatible to arequested earlier version data, and the data is determined to beunavailable for the requester for the earlier version data.

With regard to providing data to an affiliated device, embodiments ofthe invention provide a variety of approaches. FIG. 15A is an exemplaryillustration of data distribution in a movable object environmentaccording to one embodiment of the invention.

At reference 1502, the movable object 550 receives a request forcharacteristic data from a requester the affiliated device 502. Atreference 1504, the movable object 550 confirms that that thecharacteristic data is available on the movable object and that therequester has a privilege to access the characteristic data. Then atreference 1508, the movable object 550 provides the characteristic datato the affiliated device 502 responsive to the confirmation. The datamay be provided at one time based on the request for characteristicdata. In alternative the data may be provided continuously at aninterval based on the request, in which case the request may be referredto as a data subscription request. As illustrated at reference 1502 inFIG. 15A, the data subscription interval is 500 milliseconds in thisexample. The data subscription interval is often in the range ofmilliseconds to seconds. For time sensitive data, the data subscriptioninterval may be in the range of 5 milliseconds to 500 milliseconds(2-200 Hz in the frequency domain); for non-time sensitive data, thedata subscription interval may be in the range of 500 milliseconds to 10seconds (0.1-2 Hz). Obviously, other data subscription interval rangesmay be implemented depending on the criticality and practicality of datasubscription.

For a data subscription request, the interval of the data transmissionto the affiliated device 502 may be based on information indicative ofthe required interval in the data subscription request. For example, therequest may specify the interval for data distribution. Alternatively,the interval may be based on the required data type. While the datasubscription request itself does not indicate an interval, the movableobject 550 may determine a suitable interval of the required data typeto transmit the required characteristic data. For example, the movableobject may determine the data subscription interval for movable objectposition to be 20 milliseconds as real-time positional data is timecritical for the affiliated device's control over the movable object,yet the data subscription interval for movable object’ available datastorage space to be ten seconds as the available data storage space isnot critical for the control over the movable object.

Additionally, the interval of the data transmission may be updated. Forexample, the affiliated device 502 may send an updated request for thecharacteristic data with information indicative of a differentsubscription interval, and based on the updated request, the datasubscription interval can be updated.

Furthermore, the movable object 505 may receive another request foranother characteristic data of the movable object from the requester oranother requester of the affiliated device 502. The movable object 505then confirms that the other characteristic data is available on themovable object and that the requester or the other requester has aprivilege to access the other characteristic data, and provide the othercharacteristic data instead of the characteristic data responsive to theconfirmations.

FIG. 15B is another exemplary illustration of data distribution in amovable object environment according to one embodiment of the invention.At reference 1512, the movable object 550 receives a request forcharacteristic data from a requester the affiliated device 504. Atreference 1514, the movable object 550 confirms that that thecharacteristic data is available on the movable object 550 and that therequester has a privilege to access the characteristic data. Then atreference 1518, the movable object 550 provides the characteristic datato the affiliated device 504 responsive to the confirmation.Additionally, the movable object 550 also provides the characteristicdata to another affiliated device 502.

That is, once the movable object 550 confirms that the characteristicdata is available on the movable object and that the requester has aprivilege to access the characteristic data, the movable object 550 maybroadcast (i.e., to any affiliated device) or multicast (i.e., to a setof selected affiliated devices) the characteristic data. The choice ofbroadcasting or multicasting of the characteristic data may be based onthe request at reference 1512, which includes information indicative ofsuch choice, and in case of multicast, the criteria to select theaffiliated devices to multicast. The choice may also be based on therequired data type. While the data request itself at reference 1512 doesnot indicate whether to provide data only to the requesting affiliateddevice 504, a group or all affiliated device (including the affiliateddevice 502), the movable object 550 may determine suitabledestination(s) to provide the characteristic data. For example, themovable object 550 may determine to multicast the requested data to allthe affiliated devices with the same requester or other requesters withthe same or higher level of privilege.

FIG. 16 is a flow diagram illustrating operations at a movable objectfor data distribution of the movable object, in accordance with variousembodiments of the present invention. Method 1600 may be implemented ina movable object controller of the movable object in one embodiment ofthe invention.

At reference 1602, a request for characteristic data of a movable objectis received from a requester of an affiliated device. At reference 1604,it is confirmed that the characteristic data is available on the movableobject and that the requester has a privilege to access thecharacteristic data. Then at reference 1606, the characteristic data isprovided to the affiliated device responsive to the confirmation.

Optionally, the process continues and an update of the privilege of therequester is received at reference 1608. Then at reference 1610, othercharacteristic data is provided or provide less of the characteristicdata based on the updated privilege.

For example, the update of the privilege may be an increase of theprivilege of the requester, thus the requester is allowed to accesscharacteristic data other than what was allowed based on an earlierprivilege. In that case, the movable object provides othercharacteristic data to the affiliated device due to the elevatedprivilege level. The providing of the other characteristic data may bedetermined by the movable object controller and without an additionalrequest from the affiliated device in one embodiment.

In contrary, the update of the privilege may be a decrease of theprivilege of the requester, thus the requester is allowed to access lesscharacteristic data than what was allowed based on the earlierprivilege. In that case, the movable object may no longer providecharacteristic data that the affiliated device now has no privilege toaccess. When the privilege is reduced to the lowest level, the movableobject may stop providing characteristic data to the affiliated deviceall together. Thus, through changing the privilege of the requester(e.g., a setting change for an application in a registration databasesuch as the registration database 411, when the requester is anapplication), the movable object environment may effectively change thedata distribution of a movable object.

During data distribution from a movable object to an affiliated device,the data distribution may produce significant workload at the movableobject and/or the affiliated device, and such workload may causecongestion and affect the performance of the movable object and/or theaffiliated device. FIG. 17 is a flow diagram illustrating load balancingduring data distribution of a movable object, in accordance with variousembodiments of the present invention. Method 1700 may be implemented ina movable object controller of the movable object in one embodiment ofthe invention. Prior to the operation of method 1700, a movable objecthas characteristic data to be provided to an affiliated device.

At reference 1702, a workload of at least one of the movable object andthe affiliated device is determined. The workload may be indicated as acentral processing unit (CPU) usage, available data storage space,and/or communication link (uplink/downlink) delay at the moment for themovable object and/or the affiliated device.

At reference 1704, the time sensitivity of the characteristic data isdetermined. The characteristic data have different time sensitivitydepending on the data type in one embodiment. For example, the motiondata of the movable object is typically more time sensitive than thedata related to applications and affiliated devices, as the former maybe used for real-time control of the movable object by the affiliateddevice thus they are time sensitive (e.g., allow delays in the order ofless than a second) while the latter can tolerate a certain level ofdelay (e.g., more than a few seconds).

Then at the reference 1706, prior to the movable object transmitting thecharacteristic data to the affiliated device, a period of time isdelayed based on at least one of the workload and the time sensitivity.

For example, the workload at the movable object is determined to be highat the time, and the characteristic data can be delayed (e.g., for anunmanned aircraft, the identification data of an application may bereported a few seconds later without major impact to theperformance/controlling of the aircraft), then the characteristic datamay be transmitted several seconds later after it is identified in themovable object database. That is, the transmitted characteristic data isnear real-time (a few milliseconds later than it is available) in thisembodiment.

In an alternative embodiment, after the period of time delay, thecharacteristic data transmitted is the data sample after the time delay.In other words, the transmitted characteristic data is still inreal-time, but it is a later sample of the characteristic data. Forexample, if a movable object's position data needs to be delayed for asecond, the position data after the one second delay is provided. Thatis, workload balancing causes the movable object provide latter samplesof the characteristic data in this embodiment.

Optionally at reference 1708, an indication is received indicatingwhether the characteristic data provided to the affiliated devicesatisfies a timing requirement of the affiliated device. At reference1710, based on the indication, the period of time is adjusted for asubsequent transmission to the affiliate device.

Through workload balancing, the data distribution of the movable objectcan avoid causing congestion at the movable object and/or the affiliateddevice, and makes the data distribution of the movable object scalable,particularly when multiple affiliated devices subscribe variouscharacteristic data of a movable object.

FIG. 18 is a flow diagram illustrating operations at an affiliateddevice for data distribution of the movable object, in accordance withvarious embodiments of the present invention. Method 1800 may beimplemented in an affiliated device such as affiliated device 502.

At reference 1802, a request for characteristic data of a movable objectis transmitted, via a communication adaptor, from a requester of theaffiliated device to the movable object. The request for characteristicdata includes information indicative of whether the affiliated devicehaving a privilege to access the characteristic data. The communicationadaptor operates to convert the request from the requester to beaccepted by the movable object in one embodiment. The conversionincludes packaging the request in a format that the affiliated devicehosting the application may transmit out the messages toward the movableobject.

At reference 1804, via the communication adaptor, the characteristicdata from the movable object is received responsive to the request forthe characteristic data. Then at reference 1806, the characteristic datais provided, via the communication adaptor, to the requester of theaffiliated device. The communication adaptor operates to convert thecharacteristic data into messages in the requester (e.g., messages in anapplication).

Optionally at reference 1808, the affiliated device (e.g., therequester) determines whether the characteristic data received from themovable object satisfies a timing requirement of the affiliated device.Then at reference 1810, an indication of the determination istransmitted, via the communication adaptor, from the affiliated deviceto the movable object.

Through embodiments of the various embodiments of the present invention,a movable object may distribute its characteristic data efficiently toone or more affiliated devices, one time or periodically based on a datadistribution interval. The data distribution interval may be updated byeither the affiliated devices or the movable object. The datadistribution workload may be adjusted through workload balancing. Theoffered data distribution flexibility makes a movable object environmentmore friendly to affiliated devices and open the movable objectenvironment to new fields of usage.

In a typical movable object environment, an operator of the movableobject environment (e.g., the vendor of a movable object) provides a setof communication parameters to an affiliated device to set up aconnection with the movable object. The set of communication parametersprovided by the movable object generally uses default values with thegoal to accommodate as many different types of affiliated devices aspossible, thus the default values are generally the “lowest commondenominators” supported among the various types of affiliated devices,and the default values do not take advantage of communicationcharacteristics of a particular affiliated device and/or a particularmovable object.

In an open movable object environment, where a movable objectcommunicates with different types of affiliated devices, it is desirableto have each affiliated device to negotiate with the movable object andfind optimized communication parameters for the connection between theaffiliated device and the movable object.

FIG. 19 is an exemplary illustration of a process to update a connectionbetween a movable object and an affiliated device, in accordance withvarious embodiments of the present invention. As illustrated in FIG. 19,a movable object environment 1900 includes a movable object 550, acommunication network 590, and an affiliated device 502, all of whichare described herein above. In FIG. 19, task boxes 1-7 illustrate theorder in which operations are performed according to one embodiment ofthe invention.

At optional task box 1, the movable object 550 confirms that theaffiliated device 502 has a privilege to communicate with the movableobject 550. The confirmation is through checking an authenticationserver such as the authentication server 401 in one embodiment. Forexample, the authentication server 401 may indicate that the affiliateddevice 502 has the privilege to access the communication module 849and/or the data distribution API 808 through which to cause thecommunication module 849 to set up a connection with the movable objectin FIG. 8, thus it has the privilege to communication with the movableobject 550. In one embodiment, the confirmation is to confirm that theaffiliated device 502 has a privilege to communicate with the movableobject 550 from an application executing on the affiliated device 502through an API that allows the application provided by a vendordifferent than that of which provides the movable object 550 to interactwith the movable object 550.

After the confirmation, the movable object 550 allows the affiliateddevice 502 to establish a connection between the movable object 550 andthe affiliated device 502. Either the movable object 550 or theaffiliated device 502 may initiate or complete the establishment of theconnection. Embodiments of the invention covers differentprotocols/procedures through which the connection is established.

The connection uses a set of communication parameters, values of whichare set to default values. In one embodiment, the connection isestablished from an application executing on the affiliated device 502through an API that allows the application provided by a vendordifferent than that of which provides the movable object 550 to interactwith the movable object 550. In one embodiment, the movable object 550extracts a command to connect to connect with the movable object 550from a request in a protocol from the application, and establishes theconnection based on the command. In one embodiment, the API is aprotocol independent API discussed herein above, where the API isadapted to accept one or more commands to connect with the movableobject extracted from a second protocol, different from the firstprotocol, for connection establishment for a different affiliateddevice.

The set of communication parameters includes a bandwidth of theconnection, a packet acknowledgement timeout period, a packetretransmission interval, and a size of packet buffer in one embodiment.FIG. 22 is an exemplary illustration of a set of communicationparameters, in accordance with various embodiments of the presentinvention. Each of the movable object and affiliated device has a set ofcommunication parameters, which may be different for different movableobjects and affiliated devices. In this example, communicationparameters 2202 have a range of values 2204 and default values 2206. Forexample, the connection bandwidth in kilobyte per second at reference2210 is in the range of 5-20 KB/s and the default value is 5 KB/s, thepacket buffer size in bytes at reference 2212 is in the range of 5-100bytes and the default value is 5 bytes, the packet acknowledgmenttimeout period in milliseconds at reference 2214 is in the range of100-500 milliseconds and the default value is 500 milliseconds, and thetransmission retry limit is in the range of 3-6 times and the defaultvalue is 3 times.

For different connections (e.g., wireline connection between the movableobject and an onboard electronic device vs. wireless connection betweenthe movable object and a wearable electronic device, a handheldelectronic device, or another movable object), the sets of communicationparameters may be different. The default values for the set ofcommunication parameters may be determined by an operator (e.g., themovable object vendor) of the movable object environment, and the goalof using the default values is to have a set of acceptable communicationparameters for various types of affiliated devices so that moreaffiliated devices may communicate with the movable object. The defaultvalues thus are “safe” values, as illustrated in FIG. 22, the defaultvalues are for a slower, less capable connection, and/or consuming lessprocessing power (e.g., the default bandwidth being at the lowest end 5KB/s and the timeout period being at the highest end 500 millisecond).The wide breadth of allowable affiliated devices for connectiontypically comes with a cost of suboptimal connection, and the defaultvalues for the set of communication parameters are often not the ones toprovide an optimal connection between a particular affiliated device anda particular movable object.

Referring back to FIG. 19, at task box 2, the movable object 550provides information for a set of communication parameters that areadapted to be used by different affiliated devices to communicate withthe movable object. The information includes values of the set ofcommunication parameters (or an encrypted version of the values)supported by the movable object 550, such as the ranges of values and/ordefault values, in one embodiment. The information may include productinformation that an affiliated device may use to determine the supportedvalues of the set of communication parameters by the movable object 550in one embodiment. For example, the information may be the serialnumber, model number, or organizationally unique identifier (OUI) of themovable object 550, from which an affiliated device may deduce thesupported values of the set of communication parameters by the movableobject 550 as the affiliated device may find the supported values fromother sources.

At task box 3, the affiliated device 502 obtains a modified value for atleast one of the set of communication parameters, based on values forthe set of communication parameters provided by the movable object andvalues for the set of communication parameters provided by theaffiliated device. Then at task box 4, the movable object 550 obtainsfrom the affiliated device 502 the modified value for the at least oneof the set of communication parameters. In one embodiment, the modifiedvalue comparing to a corresponding default value for the at least one ofthe set of communication parameter indicates at least one of a biggerbandwidth of the connection, a shorter packet acknowledgment timeoutperiod, a shorter packet retransmission interval, and a bigger size of apacket buffer.

At task box 5, the movable object 550 configures the connectionestablished between the movable object and the affiliated device basedon the modified value for the at least one of the set of communicationparameters.

Optionally at task box 6, the movable object 550 reports a status of themovable object to the affiliated device 502 through the connection. Attask box 7, when the status indicates an anomaly, the affiliated device502 causes the movable object 550 to enter a safety mode. The affiliateddevice 502 issues a request to the movable object based on the status tocause the movable object 550 to enter the safety mode in one embodiment.

It is to be noted that the modified value at task box 3 may be atemporary modification that is applicable to only the current connectionbetween the affiliated device 502 and the movable object 550. Once theconnection is terminated, for the next connection established betweenthe affiliated device 502 and the movable object 550, the default valuesof the set of communication parameters may be used initially, andoperations in the task box 1-5 may be repeated to obtain the same ordifferent modified value for the next connection.

In an alternative embodiment, the modified value at task box 3 is apermanent modification that is applicable to all subsequent connectionsbetween the affiliated device 502 and the movable object 550(unless/until the affiliated device 502 or the movable object 550 causesthe change of the modified value). In that case, a subsequent connectionwill be established using the modified value, and the modified value maybe updated again (temporarily or permanently as explained) through theoperations in the task box 1-5.

The safety mode may have several levels, based on the severity of theanomaly. For example, when the movable object 550 is an unmannedaircraft, the safety mode may be return-to-home (RTH). When the anomalyis severe and the affiliated device 502 determines that it has lostcommunication with the movable object 550 for a period of time based ona high bit error rate (BER) of the communication links between themovable object 550 and the affiliated device 502 as reported by themovable object 550, the affiliated device 502 may cause the movableobject to enter a smart RTH mode or the movable object may enter thesmart RTH mode on its own based on the loss of communication for theperiod of time, where the movable object automatically returns to thelast recorded home point that the unmanned aircraft initiates its flightfrom. When the anomaly is less severe and the status indicates that thebattery level of the movable object 550 is getting low, the affiliateddevice 502 may cause the movable object to enter a low battery RTH,where the affiliated device 502 may, at the direction of the affiliateddevice 502, take actions to cause the movable object 550 to save batteryor finish a task at hand for a period of time (e.g., 10 seconds), afterwhich the movable object 550 will automatically return to the lastrecorded home point.

It is to be noted that once the status of the movable object 550 isreceived at the affiliated device 502, in addition to or in alternativeto the affiliated device 502 causing the movable object entering thesafety mode, the affiliated device 502 may transmit the status of themovable object 550 to another electronic device. For example, the statusof the movable object 550 (anomaly or normal) may be transmitted to amanagement system (e.g., one that included in a remote controller of themovable object 550), which controls the movable object 550 based on thestatus.

As illustrated in FIG. 19, the affiliated device 502 obtains themodified value for the at least one of the set of communicationparameters and then transmits to the movable object 550, whichconfigures the connection. This approach is only one embodiment forobtaining the modified value. The negotiation of the modified value andconfigure the connection may be performed in a variety of ways, inaccordance with various embodiments of the present invention.

FIG. 20A is an exemplary illustration of communication parameternegotiation between a movable object and an affiliated device andconfigure a connection according to one embodiment of the invention.

At reference 2002, the affiliated device 502 receives information for aset of communication parameters that are adapted to be used by differentaffiliated devices to communicate with the movable object 550.

At reference 2004, the affiliated device 502 compares the receivedinformation for the set of communication parameters to the correspondingvalues for the set of communication parameters provided by theaffiliated device 502.

At reference 2006, the affiliated device 502 determines a modified valuefor at least one of the set of the communication parameters. Theaffiliated device 502 may support a range of values (including thedefault values) for each of the set of communication parameters, similarto what is discussed in relation to FIG. 22. The ranges of values may bethe same for some of the parameters and different for other parameters.The affiliated device 502 determine the modified value from the rangesof values for the set of communication parameters provided by themovable object 550 and the affiliated device 502. The modified value maybe one that is supported by both the movable object 550 and theaffiliated device 502 and that provides a better connection than thedefault value (e.g., a bigger bandwidth of the connection, a shorterpacket acknowledgment timeout period, a shorter packet retransmissioninterval, and a bigger size of a packet buffer).

It is to be noted that optionally, the affiliated device 502 and themovable object 550 may perform one or multiple rounds of negotiationsbefore the affiliated device 502 determines the modified value. Forexample, after receiving the information for the set of communicationparameters from the movable object 550, the affiliated device 502 maydetermine that for one communication parameter, several values (concretesample points, such as a packet buffer size, in bytes, being 16, 32, and64) or a narrower range of values (such as a packet buffer size rangebeing 16 to 64) within the range of the values supported by the movableobject 550 are preferable. The affiliated device may transmit theseveral values or the narrower range of values to the movable object550, which support a wider range of value (such as a packet buffer sizerange being 5-100). The movable object 550 may examine the now narrowedselection, and determine one value, a few values, or even narrower rangeof values for the parameter, and send back to the affiliated device 502.The affiliated device 502 may start operations in reference 2006 anddetermine the modified value. Or the affiliated device 502 may initiateanother round of negotiation with the movable object 550. The parametervalue negotiation such as the negotiation 2005 makes the modified valuemore likely to be optimal for both the affiliated device 502 and themovable object 550.

In one embodiment, the modified value may not be for a communicationparameter whose information is provided by the movable object. Forexample, the information provided by the movable object 550 includes asize of its packet buffer (e.g., the supported range of its packetbuffer size). The affiliated device 502 may determine that the size ofthe packet buffer is big enough to support a shorter packetacknowledgment timeout period for the connection (the bigger the packetbuffer, the more packets can be buffered for retransmission thus theacknowledgement timeout period can be shorter resulting in a fasterretransmission), thus the provided size of packet buffer results in amodified value in the shorter packet acknowledgement timeout period.

Then at reference 2008, the modified value is transmitted to the movableobject 550. At reference 2009, the movable object 550 configures theconnection between the movable object 550 and the affiliated device 502.In one embodiment, another connection between the movable object 550 andanother affiliated device is configured, based on the modified value.For configuring the other connection between the movable object 550 andthe other affiliated device, the movable object 550 may determine thatthe affiliated device 502 is the same type as the other affiliateddevice thus the other affiliated device supports the modified value too.In this way, a value negotiated between the affiliated device 502 andthe movable object 550 may be used for configuring another connectionbetween the movable object 550 and the other affiliated device.

In one embodiment, the configured connection does not use the modifiedvalue but uses a different value instead. For example, the configurationof the connection is further based on an operational state of themovable object 550. The operational state of the movable object includesits current workload (e.g., a central processing unit (CPU) usage,and/or available data storage space), battery usage/power consumption,communication link (uplink/downlink) delay, high priority tasks, etc.For example, the movable object 550 may determine that its workload isheavy, and while the received modified value for a connection bandwidthis 20 KB/s from the affiliated device 502, the movable object 550determines to configure the connection uses a lower connection bandwidthat 15 KB/s, which is still higher than the default 5 KB/s but is moresuitable for the operation of movable object 550.

FIG. 20B is another exemplary illustration of communication parameternegotiation between a movable object and an affiliated device andconfigure a connection according to one embodiment of the invention.

The operations in optional reference 2002 is the same as in illustratedin FIG. 20A, where the affiliated device 502 receives information for aset (referred to as the first set) of communication parameters that areadapted to be used by different affiliated devices to communicate withthe movable object 550. However, in FIG. 20B, the affiliated device 502provides information for a second set of communication parameters thatare adapted to be used by the movable object 550 to communicate with theaffiliated device 502. The first and second sets of communicationparameters may be the same or different. In one embodiment, theaffiliated device 502 determines the second set of the communicationparameters whose information to be provided based on the first set ofcommunication parameters the affiliated device 502 receives. Forexample, when the first set of communication parameters includes apacket acknowledgment timeout period, the affiliated device 502 willsend information for both the packet acknowledgment timeout period andthe size of packet buffer provided by the affiliated device 502 as thetwo parameters are related (e.g., a short packet acknowledgment timeoutperiod requires a large size of packet buffer). Conversely, the secondset of communication parameters may contain less communicationparameters than the first set (e.g., because the affiliated device 502determines that the movable object 550 has the information already).

In one embodiment, without the movable object 550 providing theinformation for the set of communication parameters first, theaffiliated device 2012 provides information for a set of communicationparameters to the movable object 550.

At reference 2014, the movable object 550 compares the receivedinformation for the set of communication parameters from the affiliateddevice 502 to the corresponding values for the set of communicationparameters provided by the movable object 550. At reference 2016, themovable object 550 determines a modified value for at least one of theset of the communication parameters. The operations in references 2014,2015, and 2016 are similar to those in references 2004, 2005, and 2006respectively, except the former are performed by the movable object 550and the latter are performed by the affiliated device 502. Then atreference 2018, the modified value is transmitted to the affiliateddevice 502.

At reference 2019, the affiliated device 502 configures the connectionbetween the movable object 550 and the affiliated device 502 based onthe modified value. It is to be noted in contrast to FIG. 20A, theaffiliated device 502 (instead of the movable object 550) configures theconnection. In other words, both an affiliated device and a movableobject may configure the connect based on the modified value. Similar tothe movable object 550, affiliated device 502 does not use the modifiedvalue to configure the connection but uses a different value instead inone embodiment. Instead, the configuration of the connection is furtherbased on an operational state of the affiliated device 502. Theoperational state of the movable object includes its current workload(e.g., central processing unit (CPU) usage, available data storagespace), communication link (uplink/downlink) delay, power consumption,high priority tasks, etc.

It is to be noted that while one modified value is discussed forcommunication parameter negotiation, multiple modified values may benegotiation through the same process. For example, operations inreference 2006 and 2016 may determine a plurality of modified values fora plurality of communication parameters.

Once the communication parameter negotiation between a movable objectand an affiliated device is complete, the next step is to configure theconnection between the movable object and the affiliated device based onone or more modified values and use the connection to communicate themovable object's status.

FIG. 21 is an exemplary illustration of configuring and using aconnection between a movable object and an affiliated device accordingto one embodiment of the invention. The configuration of the connectionoccurs after one or more modified values for communication parametershave been obtained.

Optionally at reference 2102, either the movable object 550 or theaffiliated device 502 terminates the connection that communicationparameters of which uses default values. Then after the termination,either the movable object 550 or the affiliated device 502 reestablishesthe connection with the communication parameters using the one or moremodified values at reference 2104. In one embodiment, other than the oneor more modified values, the rest of the set of communication parametersuses default values. It is to be noted that in an alternativeembodiment, the configuration of the connection does not involvetermination and re-establishing the connection, rather, theconfiguration is to update one or more the communication parameters ofthe connection.

Then at reference 2106, the movable object 550 reports a status of themovable object to the affiliated device 502 (e.g., high BER and/or lowbattery low as discussed herein above). Upon receiving the status of themovable object, the affiliated device 502 may report the movable objectstatus to another electronic device at reference 2017. For example, themovable object status may be reported to a handheld electronic device ora wearable electronic device such as a remote controller of the movableobject 550, and the other electronic device may examine the movableobject status and determine the proper response based on the status.Additionally or alternatively at reference 2108, the affiliated device502 itself may determine a response to the movable object status. Thenat reference 2110, the affiliated device issues a request to the movableobject 550 based on the determination. Once the movable object 550receives the request, the movable object 550 responds to the requestfrom the affiliated device 502 (e.g., entering into a safety mode) atreference 2112.

FIG. 23 is a flow diagram illustrating the operations of a movableobject for automatic update of a connection to the movable object inmovable object environment, in accordance with various embodiments ofthe present invention. Method 2300 may be performed by a movable objectcontroller (e.g., the movable object controller 621 or 821 in FIGS. 6-8)of a movable object.

Optionally at reference 2302, it is confirmed that an affiliated devicehas a privilege to communicate with the movable object from anapplication executing on the affiliated device through an applicationprogramming interface (API) that allows the application provided by avendor different than that of which provides the movable object tointeract with the movable object. In one embodiment, the confirmationincludes checking a registration of the affiliated device. In oneembodiment, the registration of the affiliated device is checked throughan authentication server of the movable object in one embodiment.

At reference 2304, information for a set of communication parametersthat are adapted to be used by different affiliated devices tocommunicate with a movable object is provided. At reference 2306, amodified value for at least one of the set of communication parametersis obtained. Then at reference 2308, a connection established betweenthe movable object and the affiliated device is configured based on themodified value for the at least one of the set of communicationparameters.

Optionally at reference 2310, a status of the movable object is reportedto the affiliated device through the connection. The status indicates ananomaly in one embodiment, and at reference 2312, the movable objectenters a safety mode responsive to a request issued by the affiliateddevice based on the status.

FIG. 24 is a flow diagram illustrating the operations of an affiliateddevice for automatic update of a connection to the movable object inmovable object environment, in accordance with various embodiments ofthe present invention. Method 2400 may be performed by an application ofthe affiliated device.

At reference 2402, a connection is established between a movable objectand an affiliated device, where the connection uses a set ofcommunication parameters, values of which are set to default values. Inone embodiment, the connection is established from an applicationexecuting on the affiliated device through an API that allows theapplication provided by a vendor different than that of which providesthe movable object to interact with the movable object.

At reference 2404, a modified value for at least one of the set ofcommunication parameters is obtained, based on values for the set ofcommunication parameters provided by the movable object and values forthe set of communication parameters provided by the affiliated device.

At reference 2406, the connection is configured based on the modifiedvalue for the at least one of the set of communication parameters. Inone embodiment, the connection is configured additionally based on anoperational state of the affiliated device, and the connection isconfigured using a value different from the modified value for the atleast one of the set of communication parameters.

Through embodiments of the various embodiments of the present invention,a movable object and an affiliated device may automatically update aconnection established between the movable object and the affiliateddevice. The negotiation of a set of communication parameters for theconnection allows optimization of the connection based on thecommunication parameters provided by the movable object and theaffiliated device, and the movable object may report it statuses to theaffiliated device, which may control the movable object accordingly.

FIG. 25 is an exemplary illustration of a movable object, in accordancewith various embodiments of the present invention. The movable object2500 is an electronic device including many different components. Thesecomponents can be implemented as integrated circuits (ICs), portionsthereof, discrete electronic devices, or other modules adapted to acircuit board such as a motherboard or add-in card of a computingsystem, or as components otherwise incorporated within a chassis of thecomputing system. Note also that the movable object 2500 is intended toshow a high level view of many components of the computing system.However, it is to be understood that additional components may bepresent in certain implementations and furthermore, differentarrangement of the components shown may occur in other implementations.

In one embodiment, the movable object 2500 includes one or moremicroprocessors 2501, propulsion unit 2502, and non-transitorymachine-readable storage medium 2503, and optional devices 2504-2508that are interconnected via a bus or an interconnect 2510. The one ormore microprocessor 2501 represent one or more general-purposemicroprocessors such as a central processing unit (CPU), or processingdevice. More particularly, the microprocessor 2501 may be a complexinstruction set computing (CISC) microprocessor, reduced instruction setcomputing (RISC) microprocessor, very long instruction word (VLIW)microprocessor, or microprocessor implementing other instruction sets,or microprocessors implementing a combination of instruction sets.Microprocessor 2501 may also be one or more special-purpose processorssuch as an application specific integrated circuit (ASIC), a cellular orbaseband processor, a field programmable gate array (FPGA), a digitalsignal processor (DSP), a network processor, a graphics processor, anetwork processor, a communications processor, a cryptographicprocessor, a co-processor, an embedded processor, or any other type oflogic capable of processing instructions.

The one or more microprocessor 2501 may communicate with non-transitorymachine-readable storage medium 2503 (also called computer-readablestorage medium), such as magnetic disks, optical disks, read only memory(ROM), flash memory devices, and phase change memory. The non-transitorymachine-readable storage medium 2503 may store information includingsequences of instructions, such as computer programs, that are executedby the one or more microprocessors 2501, or any other device units. Forexample, executable code and/or data of a variety of operating systems,device drivers, firmware (e.g., input output basic system or BIOS),and/or applications can be loaded in the one or more microprocessor 2601and executed by the one or more microprocessor 2601.

The non-transitory machine-readable storage medium 2503 may store anauthentication unit 2523, a movable object controller 2522, and amovable object database 2525, each of which contain instructions and/orinformation to perform operations discussed herein above with regard tothe corresponding authentication units (e.g., authentication units 423or 623), movable object controller (e.g., movable object controller 621or 821), and movable object database (e.g., movable object database625). The movable object controller 2522, along with the authenticationunit 2523 and the movable object database 2525 may operate to performoperations discussed herein above in methods 900, 1000, 1300, 1600,1700, and 2300 in accordance with various embodiments of the presentinvention. The non-transitory machine-readable storage medium 2503 mayalso store computer program code, executable by the one or moremicroprocessor 2501, to perform operations discussed herein above inmethods 900, 1000, 1300, 1600, 1700, and 2300 in accordance with variousembodiments of the present invention.

The propulsion unit 2502 may include one or more devices or systemsoperable to generate forces for sustaining controlled movement of themovable object 2500. The propulsion unit 2502 may share or may eachseparately include or be operatively connected to a power source, suchas a motor (e.g., an electric motor, hydraulic motor, pneumatic motor,etc.), an engine (e.g., an internal combustion engine, a turbine engine,etc.), a battery bank, etc., or combinations thereof. The propulsionunit 2502 may also include one or more rotary components connected tothe power source and configured to participate in the generation offorces for sustaining controlled flight. For instance, rotary componentsmay include rotors, propellers, blades, nozzles, etc., which may bedriven on or by a shaft, axle, wheel, hydraulic system, pneumaticsystem, or other component or system configured to transfer power fromthe power source. The propulsion unit 2502 and/or rotary components maybe adjustable with respect to each other and/or with respect to movableobject 2500. The propulsion unit 2502 may be configured to propelmovable object 2500 in one or more vertical and horizontal directionsand to allow movable object 2500 to rotate about one or more axes. Thatis, the propulsion unit 2502 may be configured to provide lift and/orthrust for creating and maintaining translational and rotationalmovements of movable object 2500.

The movable object 2500 may optionally further include display controland/or display device unit 2504, wireless transceiver(s) 2505, video I/Odevice unit(s) 2506, audio I/O device unit(s) 2507, and other I/O deviceunits 2508 as illustrated. The wireless transceiver 2505 may be a WiFitransceiver, an infrared transceiver, a Bluetooth transceiver, a WiMaxtransceiver, a wireless cellular telephony transceiver, a satellitetransceiver (e.g., a global positioning system (GPS) transceiver), orother radio frequency (RF) transceivers, or a combination thereof.

The video I/O device unit 2506 may include an imaging processingsubsystem (e.g., a camera), which may include an optical sensor, such asa charged coupled device (CCD) or a complementary metal-oxidesemiconductor (CMOS) optical sensor, utilized to facilitate camerafunctions, such as recording photographs and video clips andconferencing. The video I/O device unit 2506 may be a 4Kcamera/camcorder in one embodiment.

An audio I/O device unit 2507 may include a speaker and/or a microphoneto facilitate voice-enabled functions, such as voice recognition, voicereplication, digital recording, and/or telephony functions. Otheroptional devices 2508 may include a storage device (e.g., a hard drive,a flash memory device), universal serial bus (USB) port(s), parallelport(s), serial port(s), a printer, a network interface, a bus bridge(e.g., a PCI-PCI bridge), sensor(s) (e.g., a motion sensor such as anaccelerometer, gyroscope, a magnetometer, a light sensor, compass, aproximity sensor, etc.), or a combination thereof. The optional deviceunits 2508 may further include certain sensors coupled to theinterconnect 2510 via a sensor hub (not shown), while other devices suchas a thermal sensor, an altitude sensor, an accelerometer, and anambient light sensor may be controlled by an embedded controller (notshown), dependent upon the specific configuration or design of themovable object 2500.

FIG. 26 is an exemplary illustration of an affiliated device, inaccordance with various embodiments of the present invention. Asillustrated in FIG. 26, an affiliated device 2600 is similar to themovable object 2500, and differences between the two are discussedherein. The propulsion unit 2602 is an optional module in the affiliateddevice 2600, which may be implemented when the affiliated device 2600 isanother movable object.

The non-transitory machine-readable storage medium 2603 may store acommunication adaptor 2622 and an application 2612, each of whichcontain instructions and/or information to perform operations discussedherein above with regard to the corresponding communication adaptor(e.g., communication adaptors 532, 732-734, or 832-836), and application(e.g., application 112, 204, 304, 314, 404, or 512-516). Thecommunication adaptor 2622 and application 2612 may operate to performoperations discussed herein above in methods 1800 and 2400 in accordancewith various embodiments of the present invention. The non-transitorymachine-readable storage medium 2603 may also store computer programcode, executable by the one or more microprocessor 2601, to performoperations discussed herein above in methods 1800 and 2400 in accordancewith various embodiments of the present invention.

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample, and not limitation. It will be apparent to persons skilled inthe relevant art that various changes in form and detail can be madetherein without departing from the spirit and scope of the invention.

The present invention has been described above with the aid offunctional building blocks illustrating the performance of specifiedfunctions and relationships thereof. The boundaries of these functionalbuilding blocks have often been arbitrarily defined herein for theconvenience of the description. Alternate boundaries can be defined solong as the specified functions and relationships thereof areappropriately performed. Any such alternate boundaries are thus withinthe scope and spirit of the invention.

The foregoing description of the present invention has been provided forthe purposes of illustration and description. It is not intended to beexhaustive or to limit the invention to the precise forms disclosed. Thebreadth and scope of the present invention should not be limited by anyof the above-described exemplary embodiments. Many modifications andvariations will be apparent to the practitioner skilled in the art. Themodifications and variations include any relevant combination of thedisclosed features. The embodiments were chosen and described in orderto best explain the principles of the invention and its practicalapplication, thereby enabling others skilled in the art to understandthe invention for various embodiments and with various modificationsthat are suited to the particular use contemplated. It is intended thatthe scope of the invention be defined by the following claims and theirequivalence.

1. A method for supporting communication in a movable objectenvironment, comprising: providing information for a set ofcommunication parameters that are adapted to be used by differentaffiliated devices to communicate with a movable object; obtaining froman affiliated device a modified value for at least one of the set ofcommunication parameters; and configuring a connection establishedbetween the movable object and the affiliated device based on themodified value for the at least one of the set of communicationparameters.
 2. The method of claim 1, wherein the movable object is anunmanned aircraft, an unmanned vehicle, or a robot.
 3. The method ofclaim 1, wherein the affiliated device is an onboard electronic deviceon the movable object, and wherein the set of communication parametersis for a wireline connection between the onboard electronic device andthe movable object.
 4. The method of claim 1, wherein the affiliateddevice is a wearable electronic device, a handheld electronic device, orone other movable object, and wherein the set of communicationparameters is for a wireless connection between the wearable electronicdevice, the handheld electronic device, or the other movable object andthe movable object.
 5. The method of claim 1, wherein the configurationof the connection is further based on an operational state of themovable object, and wherein the connection is configured using a valuedifferent from the modified value for the at least one of the set ofcommunication parameters.
 6. The method of claim 1, wherein theconnection utilizes default values for the set of communicationparameters except the modified value for the at least one of the set ofcommunication parameters.
 7. The method of claim 1, wherein the modifiedvalue for the at least one of the set of communication parameters isutilized to configure another connection between the movable object andanother affiliated device.
 8. The method of claim 1, wherein the set ofcommunication parameters includes at least one of: a bandwidth of theconnection; a packet acknowledgement timeout period; a packetretransmission interval; and a size of packet buffer.
 9. The method ofclaim 8, further comprising: reporting a status of the movable object tothe affiliated device through the connection.
 10. The method of claim 9,wherein the status indicates an anomaly, and wherein the method furthercomprises: entering a safety mode responsive to a request issued by theaffiliated device based on the status.
 11. The method of claim 1,wherein prior to providing the information for the set of communicationparameters that are to be used by the affiliated device to communicatewith the movable object, the method further comprising: confirming thatthe affiliated device has a privilege to communicate with the movableobject from an application executing on the affiliated device through anapplication programming interface (API) that allows the applicationprovided by a vendor different than that of which provides the movableobject to interact with the movable object.
 12. The method of claim 11,wherein the confirmation includes checking a registration of theaffiliated device.
 13. The method of claim 12, wherein the registrationof the affiliated device is to be checked through an authenticationserver of the movable object.
 14. A system for supporting communicationin a movable object environment, comprising: one or moremicroprocessors; and a controller, running on the one or moremicroprocessors, wherein the controller operates to provide informationfor a set of communication parameters that are to be used by differentaffiliated devices to communicate with a movable object, obtain from anaffiliated device a modified value for at least one of the set ofcommunication parameters, and configure a connection established betweenthe movable object and the affiliated device based on the modified valuefor the at least one of the set of communication parameters.
 15. Thesystem of claim 14, wherein the movable object is an unmanned aircraft,an unmanned vehicle, or a robot.
 16. The system of claim 14, wherein theaffiliated device is an onboard electronic device on the movable object,and wherein the set of communication parameters is for a wirelineconnection between the onboard electronic device and the movable object.17. The system of claim 14, wherein the affiliated device is a wearableelectronic device, a handheld electronic device, or one other movableobject, and wherein the set of communication parameters is for awireless connection between the wearable electronic device, the handheldelectronic device, or the other movable object and the movable object.18. The system of claim 14, wherein the configuration of the connectionis further based on an operational state of the movable object, andwherein the connection is configured using a value different from themodified value for the at least one of the set of communicationparameters.
 19. The system of claim 14, wherein the connection is toutilize default values for the set of communication parameters exceptthe modified value for the at least one of the set of communicationparameters. 20-26. (canceled)
 27. A non-transitory computer-readablestorage medium with instructions stored thereon, that when executed by aprocessor, perform operations for supporting communication in a movableobject environment, the operations comprising: providing information fora set of communication parameters that are adapted to be used bydifferent affiliated devices to communicate with a movable object;obtaining from an affiliated device a modified value for at least one ofthe set of communication parameters; and configuring a connectionestablished between the movable object and the affiliated device basedon the modified value for the at least one of the set of communicationparameters. 28-59. (canceled)